Markov Decision Processes

Find the course details here. Registration is open till 26/01/2024.

  • Module-1A Foundations of Probability.
  • Module-1B Stochastic processes and DTMC.
  • Module-1C Foundations of Optimization.
  • Module-2A Introduction to Markov Decision Processes.
  • Module-2B Finite-Horizon Markov Decision Processes.
  • Module-2C Infinite-Horizon Discounted Markov Decision Processes.
  • Module-2D Infinite-Horizon Average Reward Markov Decision Processes.
  • Module-3A Constrained Markov Decision Processes.
  • Module-3B Partially Observable Markov Decision Processes.

    • Random Processes (E2 202)

  • 08 Aug 2022: Tutorial-01 Cardinality. Probability space. Limits of sets. Continuity of probability.
  • 15 Aug 2022: Tutorial-02 Law of total probability. Independence of events, Conditional probability. Random variables.
  • 22 Aug 2022: Tutorial-03 Random vectors. Transformation of random variables and random vectors.
  • 29 Aug 2022: Tutorial-04 Random processes. Expectations.
  • 05 Sep 2022: Tutorial-05 Moments. Correlation. Lp space. Inequalities.
  • 12 Sep 2022: Tutorial-06 Generating functions. Conditional Expectation.
  • 19 Sep 2022: Tutorial-07 Almost Sure Convergence. Convergence in Probability. Borel-Cantelli Lemma.
  • 24 Oct 2022: Tutorial-08 Lp Convergence. Convergence in Distribution. Problems on Convergence of RV.
  • 26 Sep 2022: Tutorial-09 Law of Large Numbers. Central limit theorem.
  • 10 Oct 2022: Tutorial-10 Introduction to Discrete Time Markov chains.
  • 17 Oct 2022: Tutorial-11 DTMC: Recurrent and transient states.
  • 24 Oct 2022: Tutorial-12 DTMC: Communicating classes. Invariant distribution.